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1 Introduction 

Brain's activity results from the interplay of an extraordinary large number of neurons gathering 
into large-scale populations. This view is supported by anatomical evidences: neurons with the same 
dynamics and interconnection properties gather into cortical columns {neural populations) with a 
diameter of about bOfim to 1mm, containing of the order of few thousands to one hundred thousand 
neurons in charge of specific functions [T7] . This macroscopic signal emerging from the interaction of a 
large number of cells is recorded by most non- invasive imaging techniques (EEG /MEG / Optical Imaging 
and Local Field Potentials). These columns have specific functions and spatial locations resulting in 
the presence of delays in their interactions due to the transport of information through axons and to 
the typical time the synaptic machinery needs to transmit it. These delays have a clear role in shaping 
the neuronal activity, as established by different authors (see e.g. [71[19, ). Each neuron involved in this 
processing has a nonlinear dynamics and is subject to an intense noise. 

This picture motivated the development of models of large-scale activity for stochastic neuronal 
networks 1,29,5,4J for different models. Some of the approaches are heuristic, and other based on 
statistical physics methods. Recently, extensions of the theory of mean-field limits to neuronal systems 
including spatial extension and delays have been developed in a mathematically rigorous framework |23| 
1251122] . These papers exhibited the limit as the number of neurons goes to infinity, of multi-population 
(up to a continuum) neuronal networks in the presence of external noise and delays, and showed an 
essential role of noise in the qualitative dynamics of the macroscopic activity. 

However, all these work overlook a prominent aspect of brain networks. Indeed, deeper analysis 
of the brain's connectivity evidence a high degree of heterogeneity, in particular in the interconnec- 
tions [181ll6j . These are due to a number of phenomena, among which intervene the precise number of 
receptors and the extremely slow plasticity mechanisms. All these phenomena induce a static disorder 
termed static random synaptic heterogeneities. Moreover, thermal noise, channel noise and the intrin- 
sically noisy mechanisms of release and binding of neurotransmitter llj result in stochastic variations 
of the synaptic weights termed stochastic synaptic noise. 

Experimental studies of cortical areas |2] showed that the degree of heterogeneity in the connections 
significantly impacts the input-output function, rhythmicity and synchrony. Yet, qualitative effects 
induced by connection heterogeneities are still poorly understood theoretically. One notable exception 
is the work of Sompolinsky and collaborators |20j . In the thermodynamic limit of a one-population 
firing-rate neuronal network with synaptic weights modeled as centered independent Gaussian random 
variables, they evidenced a phase transition between a stationary and a chaotic regime as the disorder 
is increased. More recently, we showed |24j that an additional phase transition towards synchronized 
activity could occur upon increase of the heterogeneity in multi-populations networks. 

The topic of the present manuscript is to analyze in a mathematically rigorous manner the effect 
of heterogeneities and noise in large-scale networks in the presence of delays. One objective of the 
present manuscript is to understand from a mathematical perspective the mean-field equations used 
by Sompolinsky and colleagues in 20J and to generalize their approach in order to be closer from the 
biological problem that motivates this study. The paper is organized as follows. In section [2] we will 
present the mathematical framework and the equations that will be studied throughout the paper, and 
in particular the inclusion of interaction delays between cells, multiple populations and non-centered 
synaptic coefficients which were not present of prior models analyzed. From a mathematical perspective, 
this problem falls in the class of interacting diffusions in a random environment and is reminiscent of the 
work developed by Ben Arous and Guionnet on the Sherrington-Kirpatrick model [3lll3j for spin-glass 
dynamics. Following their approach, we will extend their framework in order to address our biological 
problem. A heuristic physical argument and a summary of the results is provided in that section, and 
proofs will be presented in section[3]and appendix [X] Similarly to their analysis of spin glass dynamics, 
we will use large-deviation techniques to characterize the macroscopic limits of these systems in the 
limit where the number of neurons tends to infinity. There are two main technical distinctions with 
existing literature. First is the fact that we deal with multi-populations networks, which involves highly 
nontrivial difficulties, some of which will be further analyzed in a forthcoming study. Second, the fact 
that the connectivity weights are not centered introduce additional terms that need special care. And 
third is the presence of delays in the interactions, leading to work in infinite-dimensional spaces. 

The limit of the systems being found, we will analyze the solutions of the asymptotic system in 
section |4] Fortunately, we will show that the solutions of the limit system has Gaussian solutions 
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that are exponentially attractive. These Gaussian solutions are univocally characterized by their mean 
and covariance functions, and these variables satisfy a closed set of deterministic equations, similar 
to the ones generally used by physicists from the dynamic mean-field theory [20] ■ The mean satisfies 
a delayed differential equation coupled to the variance of the solution. These are extremely hard to 
study analytically. In particular, in contrast with the case of deterministic synaptic weights |251I22| . 
the variance does not satisfies an ordinary differential equation but can be written as the solution of 
a fixed point equation. This distinction is fundamentally related to the non-Markovian nature of the 
asymptotic equation. However, based on the bifurcation analysis of the equation on the mean of the 
Gaussian solution, an alternative method to more usual analyzes of these mean-field equations, we will 
be able to characterize the solutions and demonstrate that noise is directly related to the emergence of 
synchronized oscillations, a highly relevant macroscopic state related to fundamental cortical functions 
such as memory, attention, sleep and consciousness, and its impairments relate to serious pathologies 
such as epilepsy or Parkinson's disease pSlfB]. and that may account for the results of Aradi and 
colleagues showing that increased heterogeneity was related with epilepsy. 



2 Mathematical setting 

In this section we introduce the model that we will be investigating in the present manuscript and the 
main results of the analysis. A heuristic discussion is provided to explain from a physical viewpoint 
the results that we will establish in section [H 



2.1 The Random Neural Network Model 

In all the manuscript, we are working in a complete probability space (i7,J^, P) endowed with a 
filtration {J^t)^ satisfying the usual conditions. We consider a network composed of N neurons falling 
into one of M populations. We define byp:Ni— >-{l,---, M} the population function associating to 
a neuron index the population label it belongs to. The state of each neuron i in population p{i) — 
a G {1, ■ ■ • , Af} is described by its membrane potential € R, and considered to satisfy a firing-rate 
equation (similar to the Hopfield network well known to physicists, see [T UHll^ V 



where 9a is the characteristic time of neurons of population a, Jij is the synaptic efficiency of the 
synapse between neuron j and neuron i, Sa-y the sigmoidal voltage-to-rate function, Ta-y the propagation 
delay between neurons of population 7 and neurons of population a, and the noise intensity. The 
Brownian motions account for the noisy input received by all neurons, and the heterogeneity of the 
connections is integrated in the synaptic weights Jij. These weights are assumed to be independent 
Gaussian random variables, with law Af{^^^^^^^^, °'p(')p(j) -j^ g^j^j ^j^g stochastic synaptic noise case, 

these will be assumed to be stochastic processes, further specified below. Let us emphasize that all the 
results readily generalize to neurons having a nonlinear intrinsic dynamics: 



under the condition that the functions fa for a G {1, • ■ ■ , M} satisfy two standard regularity conditions: 

1. The functions fa are uniformly locally Lipschitz-continuous in their second variable. 

2. They satisfy a uniform monotone growth condition: for any < G R, xfa{t, x) < C(l + x^). 

These refinements allow considering more biologically relevant neuron models. However, generalizations 
of the interaction that would depend on both the pre- and post-synaptic neurons (consisting in replacing 




(1) 



M 



dxl = fait, Xj) + E E J^JSayixUJ dt + XadWl 
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the function Saj{xl_.j.^^ ) by a nonlinear function b{xl,xl_^^^) in equation ([!])) are harder to achieve 
and the methodology used in the present manuscript cannot be extended to these more general cases. 

Let us define r = max^^To,^, and let {fJ.a)a£{i---M} ^ collection of M probability distributions on 
Cr = C([— r, 0],R) that will be used to characterize the initial condition of neurons in population a. 
The initial condition on the network are considered chaotic, given by: 

N 

Law of {xt)te[-T,o] = (^/^p(i)- (2) 
1=1 

The function {Sa-y)a,-ye{i,--- ,m} : R R are Lipschitz-continuous sigmoidal functions, i.e. non- 
decreasing functions tending to at — oo and to 1 at oo. This model differs from the work of Ben-Arous 
and Guionnet in that the voltage is not bounded, there exist multiple populations and the interactions 
are nonlinear and delayed, which sets the problem in an infinite-dimensional space. The first question 
that may arise at this point is the well-posedness of the system. 

Proposition 1 For each J G R^^-'^ and T > 0, there exists a unique weak solution to the system ([T]) 
defined on [— r, T] with initial condition ([2|. Moreover, this solution is square integrable. 

The proof of this result direct stems of standard theory of delayed stochastic differential equations [HI 
[T5] . Indeed, the network equations correspond to delayed stochastic differential equation in dimension 
N. For these equations, it is established that existence, uniqueness and square integrability of the 
solution hold under the assumption that the drift and diffusion functions are Lipschitz-continuous and 
enjoy a linear growth property, which clearly hold for our system. 

Remark 1 Note that if the initial condition was given by (x\)t(z[-T,o\ — C with = iJ.p(^i^, we can also 
prove strong existence and uniqueness of solutions. 



2.2 Heuristic approach to mean-field equations 

The mean-field approach consists in reducing the A^-dimensional delayed stochastic differential equa- 
tion ([T]) to a Af-dimensional equation on the law of the system in the limit where all Na — oo. 
In ([ij, one needs to characterize the behavior of X^^i Sj p(j)=7 Jij'^a-ri^t-Tc-,) large Na limit. 

Following the idea of molecular chaos of Boltzmann ("Stofizahlansatz"), we may assume that all neu- 
rons behave independently and independently of the disorder. This physical ansatz is the basis of our 
heuristical approach. Assuming that the processes xl_^^_^ and the coefficients Jij are independent, the 

local interaction term J^^Li Sj p(j)=7 •^y '5'Q7(a^t-ro^) is a sum of independent gaussian processes. A 
functional version of the central limit theorem (under suitable regularity and boundedness conditions) 
would ensure that 

M I M M \ 

7=lj:p(j)=7 \7=1 7=1 / 

where the Gaussian limits are pairwise independent. In other words, under the molecular chaos hy- 
pothesis, averaging effects occur at the level of all neurons, and an effective interaction term in the 
form of a Gaussian process, will be obtained as an effective collective interaction term: 



' ^ (S7ii Joi-i'^\SaiijQ_^^^_^\ , Y!''^=\ '^a7iE[S'a7(^7-r„^)^]) pairwisc independent. 



(3) 



This is precisely the same kind of results one obtains in the mean-field theory for spin-glasses. From 
a mathematical viewpoint, the assumptions of independence of the variables and of the Jij cannot 
be rigorously legitimated. 

And unfortunately, as is the case in the theory of spin glasses, rigorous approaches up to now rely 
on relatively abstract methods using in particular large-deviations theory. This is the approach we 
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will now develop. We shall eventually note that a posteriori validation of the intuition provided by 
this heuristic argument will be obtained as a side result of our analysis: we will indeed show that a 
propagation of chaos phenomenon occurs in the limit where all Na — >■ oo, i.e. that, provided the initial 
conditions are independent identically distributed for all neurons in the same population, then finite 
subsets of neurons behave independently for all times and have the same law given by the mean-field 
equation (|3|. 



2.3 Summary of the results 

Mathematical demonstration of such convergence results for multi-populations networks are still to be 
developed in general settings, and up to our knowledge no result of this kind for multi-populations 
networks with different population size exist in the literature. In particular, Sanov's theorem, hinge 
of most studies for the convergence of large-scale networks with heterogeneities, is still to be proved 
for such systems where the elements are one of a few populations and not identically distributed. In 
order to show rigorous results on the dynamics, we will slightly specify the problem and consider that 
all populations have identical sizes n (hence, N = Mn). In that case, the analysis can be reduced 
to the analysis of one-populations systems but with M-dimensional variables, whose a E {1 ■ ■ ■ M} 
component is the voltage of one of the neurons in population a. 

We now group the N variables {x^)i=i...N into M-dimensional variables X* = (a;*° , a = 1 • • • M)i=i...„ 
where p{ia) = a, and such that for any i ^ j, ia ^ ja- The dynamics of these variables, readily deduced 
from the original network dynamics, can be written in vector form as: 

dXl = (^L-Xl + Y^^ 3, J Sixl_^)^ dt + E-dWi (4) 

where L is the diagonal matrix with coefficient {a, a) equal to — J.y is the M x M matrix with 
elements {Ji^ji^)a,i3=i---M, a.Tid S{X^_^) is the MxAf matrix with elements (01,7) given by S'^c(X(f^^^ ). 
The linear operator acts on M x M matrices by giving the diagonal (as a M-dimensional vector) of the 
classical matrix product of two matrices, so that the component a of 3ijQ S{Xl_^) is precisely equal to 
Jiaj-i^a-yixl'Lr^^)- ^ IS the diagonal matrix with diagonal element Aq and WJ — {Wj:'')a=i---M- 
We now work with an arbitrary fixed time T > and denote by Q"(J) the unique law solution 
of the network equations ^ restricted to the cr-algebra (^{Xl, 1 < i < n,—T < s < T). (Q"'(J) is a 
probability measure on C" where C is the space of continuous functions of [— t, T] with value in R^. 
When neurons are not coupled (i.e. Jij = for all the law of all variables XI are identical, 

independent, and given by the unique solution P of the one-dimensional standard SDE: 

(dXt^ LXtdt + SdWt 
l(Xo)^M° 

The uncoupled system hence has the law of the Ornstein-Uhlenbeck process. P is the law of this process 
restricted to the cr-algebra Gt = o^i^s, s < T), it is a probability measure on the space Cq of continuous 
functions of [0, T] in R*^. We will denote by Pa the law of its component a (in our case, P — 0*Li^a)- 
We want to study the behavior of the empirical law on each population: 



1 " 

An = - Sxi 

n ^ — ^ 



n 
1=1 



under Q"(J) where the coefficients are independent Gaussian with law given above. By a direct 
application of Girsanov theorem, (Q"(J) is absolutely continuous with respect to P**" and we have: 



dQ"(J) _ 
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where the prime denotes the transposition. 

The aim of the present manuscript is to prove the following: 

Theorem 1 Under Q" — lEj((Q"( J)) the law of the N -neurons system averaged over all possible con- 
figurations (realizations of the synaptic weights ( Jij)), the sequence of empirical measures fin converges 
towards 6q as n goes to infinity for some probability measure Q. 

In detail, we will show that the sequence empirical measures fin satisfies a weak large deviation 
upper bound property and is tight. Precisely, we will demonstrate that: 

Theorem 2 There exists a good rate function H such that for any compact subset K of A4i {C) where 

c = c([-T, r],R^'0- 

limsup - log (3"(/i„ e K) < - inf H. 

n—^oo n K 

Moreover, we will show a tightness result on that sequence, namely: 

Theorem 3 For any real number e > 0, there exists a compact subset Kg, such that for any integer n, 

Q"(An ^ K,) < e. 

These two results imply the convergence result provided that we characterize uniquely the minima of 

H, which will be the subject of theorem: 

Theorem 4 The good rate function H is such that: 

I. It achieves its minimal value at the probability measure Q P satisfying the implicit equation: 



dQ 
dP 



■M ^ 

^ a; 



Gt-'^'dWt°' 



1 



(G^« fdt 



where £ denotes the expectation over — (G"'*^ )a=i---M a. Gaussian process with 



M „ 

^[Cr'^l ^I^^/a^y Sa^ixt-r^^)dQ^{x) 



7=1 

and covariance: 

M „ 

f[G^^«°G2'«'] / S^fs{xt-r^^)S^0{x,^r^^)dQ^{x) 

0=1 

where 5a-y equals 1 if a — j and otherwise. 
2. This provides an implicit self- consistent equation on Q the limit distribution, which has a unique 
solution. 



This theorem will be demonstrated in section [3^ 

Eventually, we will show that the unique solution has Gaussian local equilibria (theorem [7|, with 
mean {n'^{t))a=i---M and covariance C°'^{s,t) satisfying the well-posed system of deterministic equa- 
tions: 

1 ^ 

A" W = "TT^" W + Jcifciil^^it - raj), C'^'^it - T^^,t - r„^)) (7) 

7=1 

where fa-yil^^v) — J^^ Sa-y{x) '' '^2^'' — covariance is equal to zero when j3 ^ a and: 

G"(t,s) = e-(*+^'/«4G"(0,0)+^(exp2(iAs)/e„-l)+^(72^ f T e("+'')/«" Z\;^^c(m-t„^, «-r„^)dudw 

(8) 
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is a nonlinear function of ii'{u), C'^'~'{u,v), C^iuju) 

and C^'^{v^ v). 

In other words, the solution can be written, in law, as the solution of an implicit equation: 

dK = (" + t^"'^) + Xo^dW^ (9) 

where the processes {W^) are independent Brownian motions and the processes U^'^ are Gaussian 
processes with law as as described in theorem [i] These characterizations will be very handy to 
analyze the qualitative dynamics and phase transitions of the system. 



where Z\"'^(u, w) = E 



3 Large Deviations and Mean-Field limits 

3.1 Construction of the good rate function 

The aim of this section is to identify a good rate function for the system that we will use in Theorem 
[2] to show a large deviation principle. In order to introduce our good rate function, it is convenient 
to analyze for a moment a time-discretization of the equation (section 3.1.1 ), which will expedite the 
analysis of our continuous time problem. 

3.1.1 Analysis of the discrete time system 

Given an integer fc, we define A'^ = {O = <o < < ••• < tk < tk+i = r} a partition of [0,T] and 
consider the following dynamics for the neurons of population a: 



dxl 



0„, -^t 



^7=1 S 



= sup {ti e A'^lti < t} 

[Law of «)te[-.,o] =^1"^° 



ap0)(2^t('=)-r„,o))) ^^ + ^^ 



(10) 



As in Proposition [T] this system clearly admits a unique weak solution for any J G R^^^. We 
will denote (Q^"'"(J) its restriction to the cr-algebra a{xl,l < i < N\p{i) = a,—T < s < T), and 
QN^,n ^ ]Ej((Q^='"(J)). They are both probability measures onC([-T,T],R)". By Girsanov Theorem, 
Q^°'"(J) < P;f" with: 



d(Q^°-"(J) 
dP„®" 



exp 



{ E / (^^E E J^.s^,ixi,,,_^j)dwi 

i:p{i)=a " 7-1 j:p(j)=7 

I ^ 2 ^ 

-J (a^E E J^lSo.^i^lw^rJ) dty 



For every fi E A4^{C), the relative entropy with respect to P is defined by: 



IMP) 



log^d^i ifM<^', 



otherwise . 

We introduce, for /i e Mi{C), the two following functions, respectively defined on [0,r]^ and [0,T]: 



Kit) 



^E'^"7 / 



Sa^{x]_ )d^l{x) 
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Remark that, since Sa-y takes value in [0,1], both functions are bounded: K'^{s,t) < jf- and 

P ^2 



m'^it) < ^, with ka = <^lt and J„ = TJ^^i \ Ja^ 



We now define 



(Kl{s,t) 
Klis^t) 



\ 







V ... Kff{s,t)) 

It is well known that wc can find a Af -dimensional stochastic process G — {G°')a=i---M on (]7, J^, P) 
such that, for any M x M variance-covariance matrix K on [0,r]'^, there exists a probability measure 
7if, under which G is a centered gaussian process with covariance K. We shall use the shorthand 
notation 7^ for 7^^,, and £^ the expectation under 7^. Wc now define for /i G A4^{C): 



r\ix) = ^ log 1 exp I ^ (G,, {uj) + m^{ti))' ■ (W,,^, - W,,) (x) 



^"''(a*) = ^ log (y"exp I ^ (G^(c.) + m^(iO) (W^t^^, - W^t:)(^) 

' ^ E (^^ ('^) + - to }c?7K, (^)) d^i{x) 



where Wt(2:) = ( W^{x) = + j^ds) , for a; G C. Eventually, we define the function: 

" nay a— 1---A'/ 



00 otherwise 



One can easily see that = -T"'*^, as the components of G are independent under 7^. 

We further define: 



^^{^l) = \og{ / exp 



1 



G 



where 



/exp{-i/J(G^,„H)^ 



(^)(tj))^du|d7^ 



/ Q,7G{1- 



•M} 



One can easily see that this function takes values in the M x M diagonal positive matrices. Moreover, 
we can define "ff>T,k, probability measure on ]7, such that 

nf=iexp{-i/o'(G^,.,H)'di} 

dll^T.k = r d-Jf,, 

' /miiexp{-i/;(G^,.,H)^di}d7. 

under which G is a M-dimensional centered gaussian process with covariance Kj^'^ (this Gaussian 
calculus property is proved for instance in [31 Appendix A]). 
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Proposition 2 We have: 
Proof Let 



r'=(M) = r^(M) + r^(M) 



For 7f = X;„=i ' , i e {1, 2}, it is sufficient to prove that 



{x)dii{x) 



But, 



r'^'^ti) ^J^og(^J exp I £ (a;) + m«(t('=)))dWT(a;) 

= y'log I (^exp { - ^ j\m^^{t^^^) fdt] |exp { - ^ j\Gf,,, fdt}d^^'^ 

cxp{^ m-(#))dVF-}yexp{^ G«(.) (dW" - m«(#))rfi) jdi^T,.^ 



= log <^ f 



exp(^-^ (G«(„)'rf*) }-^_^ {ml{t(''^)fdt + jj^ m^(tW)diyfdM 
+ y log I y exp y G^(., (dVFf - m^(t('=))dt)^ dT^J.'" j'^M 
Standard gaussian calculus yields: 

yexp(^y^G^(.,(dW^,"-m;(tW)dt))d7Kj>^ =exp|i y G^^,,{dW,"-m'^^{t^''^)dt)^ dj^T, 
so that, 

r«''=(/.) = rr''=(/x) 

+ yy m^(t('=))rfWrrf/x + yiogjexpjiy Q G^,,,{dW,^~m'^^{t^''^)dt)^ d7^T,.||d/z 



j j\n'^{t^^y)dW^diJi+\ j j (^j%'i,,,{dW^ -ml{t^'^y)dt)^ d^jiT,udii 



which concludes the proof. 
For all Mt{C), let: 



{x) 



2 Jo 



G4«(a;) + m,(iW) 



>d'yK^{uii) \dn{x) 
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G%idWt-m^ 



dj^T,„dfi+ / / m'^it'^''^)dWt"dn 



M 

EpQ.fe 



One can easily see that Ff; = XlaLi r^'^ ■ Let 



'/(Ai|P)-r,^(M) if/(Ai|P) <oo, 



oo 



otherwise . 

We eventually denote by d-r the Vaserstein distance on Mj^(C), compatible with the weak topology: 



dTifi,!^) = inf I y 



sup \x] -y]\'^d£,{x,y)\ 

-T<t<T;je{l....,M} ) 



the infimum being taken on the laws ^ with marginals and i^. 

We now show the following regularity properties on the introduced functions: 

Lemma 1 1. There exists a positive constant Ct, depending on T but not on n, such that: |J";f (/i) — 

r^{iy)\<CTdT{fi,i^). 

2. < I{-\P) i.e. H'' is a positive function. In particular, is finite whenever I{.\P) is. 

3. There exists real constants a < 1 and rj > such that < aI{.\P) + rj. 

4. There exists a positive constant Ct, depending on T but not on n, such that: \F^^{^i) - F^{^j.)\ < 

CT{l + I{fl\P))dT{fl,>^). 

5. Defining the following probability measure on Ai^{C): 

dQt{x)^expFl;{S^)dP{x) 

= J exp (^J^ (Gt(., +m,(tW))'- dWt{x)-^J^ ||Gt(., + m,(iW)||'di^ dj,dP{x) 

we have = I{.\Q'^), so that is lower semi-continuous on A^^(C). 

6. is a good rate function. 



This technical lemma is proved in appendix A.l 
3.L2 Analysis in continuous time 

In this section, we shall prove theorem [T] as a consequence of theorems [2] and [3j We start by extending 
the function constructed in the previous section to our continuous time setting. To this purpose, let 
us define 

F :{^ e M+{C)\I{^l\P) < 00} ^ R 

fi^ J\og(^J exp(^J^ {Gt + ni^it))' -dWt-^J^ \\Gt + ni^{t) 



dt d-f^ d^ 



F" e Mt{C)\Ii^l\P) <oo}~,-r 



log / exp 



{G? + mf,{t)ydt)d^Jdti, 
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and 

'^^^^ ^ oo otherwise 

Remark that, as for the discrete time case, 7^ — V^'^ , F 



D O 
M 



Proposition 3 1. On the compact set Kl — {ij, & Ad^ {C)\I{fi\P) < L}, F'^ converges uniformly to 
F. 

2. Vfie Kl, F"{h) ^ F^{^) + F^ifi) where 



rf (m) = log Jexp(^-^-£ Gfdt^ d^, 



^mfit)dt, 

T 



3. F < I{.\P) and 3a < 1,7? > 0| T < a/(.|P) 
4- H is a good rate function. 

Proof (i) , (ii) Following the same demonstration as in Lemmajl] (i) and (iv) , we find that there exists 
Ci^T and C2,T such that (see (28) and (34)) 







\ < CiT max ( 
1 ' 7=i,mVj 





|S'c^(a;J(fc)_^^^) 'S'Q7(2:J'(fc+p)_^^^ ) I dtdfi(x)^ 

/I 
sup \Sa-y{x]) — Saj{x2)\ dfl{x)) 
\t-s\<\A^\' ' 



)(m)| <C'2,t(/(mI^) + 1) max { \ sup \S ^^{x^^) - S ^^^{xl)? dy.(^x)\ 

7=1, M \ 7 \t-s\<\A^\ ' 



But, according to (24), we have for any a > 



sup \Sa-~f{xJ)—Sa--i(x'].)\ < /(/i|P)+log / cxp < fl sup \S a^ixl) — S a^ix^y^ \dP{x) 

|t-s|<|/ifc| J I |t-s|<|zifc| 



And by bounded convergence theorem 



lim log / exp < a sup \S a^[xj^ — S a-^ix^y. \dP{x) = Q 

J I \t-s\<\Ak\ J 

Let e > 0, choosing a — ^, it is easy to see that there exists an integer k{e) such that, for 
fc> fc(£),V7e {1,...M}: 

sup \Sa,^{x]) - Sa^{x2)\^dfiix) < (/(m|P) + l)e^ 

\t-s\<\A^\ 

Hence, for any k > k{e), any p, and any /i e K^: 

Which shows that F"'^ , P2''^j and thus F"'''' converge uniformly on K^. It is not difficult to see 
that the respective limits are F", F^ and P", which implies F^ = F^ + F2 on Kj^. Besides, as 
F^ = Ea=i ^"-'^ and F = Ea=i 

we also have the uniform convergence of F^ towards F on 
(iii) The proof is identical to Lemmgfl] (iii) and (iv) . 
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(iv) Lets show that {H < i} is a compact set. H > {1 - a)I{\P) - 77 so that I{\P) < Hence 

[H < L} C {I{\P) < Let (m,), e{H< Lf C {/(|P) < ^f. As here < f±^} 

is a compact set, there exists a subsequence {fj,p^)jn such that /ij,^^ — >■ /i as m — >■ 00. We conclude by 
stating that, as H'^ converge uniformly towards H on < jz^}, the latest inherits the lower 

semi-continuity of the firsts. Hence {H < L} is a closed set so that fi € < i} and {pp^)m 
converges in {iJ < i}. 



Lemma 2 



exp 



|nr(/<„)| 



Proof By ([6]), we have: 



dQ"(J) 



= exp 



. 4 = 1 



Applying Fubini Theorem, we find that Q" ^ P**" and: 



1=1 j = l "^0 



dg'' 

But, under Ej, the J^j are independent, so that 
dQ" 



i=i 



t-T, 



dt 



i=i L V"^o .1 -'0 



Lets show that ^ ^ • X]j=i Jjj © "^(^/-x)) 1 < t < T j- is an iV/-dimensional Gaussian process with 
covariance Kp_^{t,s)^ and mean m^^(t). In fact 

n / n M \ 

j=i V " i=i 7=1 / „=i 



= 1---M 



Lets remember that the functions ,j = 1 ■ ■ ■ N are, in this equality, seen as determinists as arguments 
of the studied density. With this in mind and because of the independence of the gaussian random 
variables Jij, we can see that the components of the above vector are gaussian processes, mutually 
independents. Moreover, the mean and covariance of the component a £ {1 • • • M} are the following: 



E 



n M 



j=l 7=1 



1 - 1 " 



7=1 



i=i 



n A I 



n M 



E 



i=i 7=1 



j = l 7=1 



2 ^ y ^a7 „ ^ ^ '^a7('^t-r„^)'^a7(''^s-Ta.^) 



" 7=1 



i=i 
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We eventually find that: 



= exp|^log(^yexp{^ (G,+m^„(t))'.dWj-^^ || + (t) ||'dt}d7M„) } 
= exp |n y" log (^y"exp { y (G* + m^„(t))' • dWj - ^ ^ Gj + m/i„(t)||"dt}d7/,„^d/i„| 



Let, for any u £ A^+(C),/i G X^(C), 



(Gt + m,(t))'-dWt 



Gi + m^(t) 



=1 1 exp I y'^lG^ + ™^(t))dW^f (G^ + m^(t))'dt|d7.d/^, 

'^^'^'^ 1 cx) otherwise . 

dQ,{x) ^ exp{r,{6,)}dP{x). 
As in Lemmajl] (v), we can show that H^, — I{\Qiy) and is therefore semi-continuous. 
Theorem 5 For any compact subset K of A4^{C), 

limsup - log (9"(An G ^) < - inf H. 

Proof Let (5 < 0. Wc can find an integer m and a family {iyi)i<i<ni of probability measure on C such 
that 

m 

2 = 1 

where B{h'i, S) = {^I^tIa^, i^i) < S} and denotes the Vaserstein distance on Mi{C). A very classical 
result (see e.g. lemma 1.2.15]), ensures that 

hm sup — log Q" (/i„ e K) < max lim sup — log Q" (/*„ £ K Ci B{i/i,S)). 

n i<i<m n 

Let V £ X+(C). Lemma [2] gives us: 

g"(A„ £Kr\B{u,6)) = 



exp 



|nr(/i„)| 



'Aneifns(i^,i5) 

= / exp|n(r(/i„)-r,(/i„))|exp|nr,(/i„)|dP^" 

But, for any probability measure v £ A^^(C), Q" := exp |nr'tx(/i„)|p®" = (Qiy)**" is a probability 
measure on (C)*^". Hence, for any conjugate exponents {p,q), 



Q"(A„ £KriB{v,d)) 



exp 



'A.ie-R'n-B(i>,<5) 
<Q:{fin(^KnB{iy,S)) 



exp 



A„eifn-B(;Ai5) 



{gn(r(A„)-r,(/i„))}dQ'; 



(11) 

We will first bound the second term of the right hand side by proving the following lemma [3j Once 
this step performed, concluding the proof amounts bounding the first term in the right hand side of 
( 11 1 using the same arguments as in [131, Lemma 3.8]. Remark that, as the space we work on remains 
a polish and the ,i = 1 ■ ■ ■ n are i.i.d under Q^, we can still resort to Sanov Theorem. 
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Lemma 3 For any real number a > 1, there exists a strictly positive real number da such that, for any 
6 < Sa, there exists a function Ca{-) m R such that lim5_>.o Ca{S) — and: 



exp 



{an(r(A„) - r,(A„))}dQi: < exp{Ca(,5)n} 



The proof of this lemma is relatively technical, and it is provided in appendix |A.2[ 

Theorem 6 (Tightness) For any real number e > 0, there exists a compact set of M^{C) such 
that, for any integer n, 



Proof The proof of this theorem consists in using the relative entropy inequality (24) and to use 
the exponential tightness of the sequence of laws P**". Indeed, defining A an arbitrary set in C" and 
applying pU to the function ^ = log(l + yields: 



log(2) + /(Q|P^") 
log(l + F®"(v4)-i) 



and the exponential tightness of P^" (see e.g. |TOJ lemma 3.2.7]) ensures that for any e > there exists 
a compact subset of M^{C) such that 



^^"(An^i^e) <exp{-^}. 



The theorem is hence proved as soon as we show that there exists a finite constant C, such that for 
any integer n, I{Q"\P^"-) < Cn. Using the expression of the relative entropy and the interchangeability 
of the neurons of the same population, we find: 



/(Q"|p45«) = n j log£^„ 
For every a G {1 • • • M} let, 









f 


Gt + m^Jt) 




la 







exp 



dQ'' (12) 



dPi 



which is a probability measure on C([— t, T],R) One can see that Q" 
gaussian computations (see Lemma 5.15]), we find that: 



After some 



rp rj~t rj-i rp 

expj^ {Gt+mlSt))dWt-\j^ (GJ'+m^Jt))'di|j = exp | ^ HnQ''")dWl--\ (i/f 

"(13) 



where 



GUf- / G^,{dWl- -ml^{s)ds) 



exp 



A 



Consequently, there exists a Q^" brownian motion B^" such that: 



Hf{Q'''')ds 
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Using (13 1 and this brownian motion in (12), we have: 

(■T 



I{Q''\P^ 



E 



+ ml {t) dtdQ'' 



< n 



< n 



^ C fT / V ft -|\2 

Ey (e^^Y^-G^y. G^(diy>-m^Js)ds)Jj +ml^\t)dtdQ" 

a=l 



dQ"" d{(E)^^aQ^-')dt + 



AS / 



fit) 



(14) 



We now will bound /"(t): 



rw<3 /fA„ 



Si.,. 



dQ 



But by Cauchy-Schwarz inequality, 



" G? GZ {dW^- - {u)du) 
Jo 



ds dQ 



<°G^x / G^dBl^ 



<"G?)' 



< ^ (see |3] Appendix A), we have by Fubini Theorem and Ito's isometry: 



yir-G^x / GyBt 
By similar arguments. 



A^-Gfx I G'^mUs)ds 



4/K[f(i'<^-^-)1} 



t2 j,2 7^2 



And, 



ylf°G^G^ 



Eventually, we proved the following inequality: 



r(0<3(^ + ^^^) + 3^ y^ r(.)d. 

So that, by Gronwall lemma. 



»pr(,)<3S(i + ^)„p{3*jf} 



Thus, (14 1 impHes: 



imPn < n Y: (3^ (1 + ^) exp {3^} + ^) 
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3.2 Identification of tfie mean-field equations 



We have seen that the empirical laws /i„ satisfy a weak large deviations upper bound with good rate 
function H. In order to identify the limit of the system, we study the minima of the functions H 
through a variational study. We will show in section 3.2.1 ;hat any minimum of all H are measures 
Q <ti P satisfying the implicit equation: 



dQ 
dP 



(15) 



We will then prove in section [3^ that there exists a unique probability measure Q satisfying ( 15 ). This 
law will be further analyzed in the next section. 



3.2.1 Variational characterization of the minima of the good rate function 

The large deviation principle ensures that the sequence of empirical measures converge, and that the 
possible limits minimize the good rate functions. We hence need to identify the minima and show that 
these are uniquely defined by equation (15). To this purpose, we start by showing that any probability 
density achieving the minimum of all the H is equivalent to P. To this end, we show the following 
technical result: 

Lemma 4 Let Q be a probability measure on C which minimizes H . Then Q P . Moreover, noting 
S = {w; |§ = 0} and 5 = P{B), we have, noting Qs = ^^f^: 

- I{Qs\P) = I{Q\P) + s5\og{s) + 0{s) 

- r{Qs)^r{Q) + 0{s) 

Proof If Q minimizes H, then necessarily I{Q\P) is finite, meaning that Q <^ P. Moreover, it is easy 
to see that: 



i{Qs\P) 



log(^)dQs 



/ (l°g(S + - log(l + s5)){'^ + sls)dP 



'dP 



dP 



} 



log(^)^dP + slog(s) / dP-\og{l + s5) 



[l{Q\P) + sS log(s) - (1 + sS) log(l + sS)^ 



dQ 

dP 



slB)dP 



which proves the first point. 

The second point is proved using standard Gaussian calculus noting that if G and V are independent 
centered Gaussian processes with covariance Kq and Ki^p, then G' = ^TJ^fj has the covariance An. . 
We hence have mQ^(i) = "^Qit)+^iBp(*^ ^ g^j^^^ ^^^^^ write: 



PiQs) = / log / cxp 



Gt + y/sVt _^ mQ{t) + smigp{t)y 



l + sS 



'■^ Gt + y^Vt ^ mg(t) +5mi^p(t) 
The exponential term is given by: 



1 



■dWt 



dt Vd-yq (K) 7ibP ]dQs 



exp 



it)) dW^ - \ 
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Using the fact that G" and are Gaussian processes with bounded covariances and using the fact 
that the mean quadratic variation of W" under Q" is also bounded (see [Sj) we can obtain that: 

r exp|Ef=i/o'(G?+mg(t))dW^f-i/„^(G?+m5(t))'dA 

/ / R{s) p ^ -^^^ ^ d^Q®lisPdQs = 0{l) 

and we finally obtain: 

r{Q,) =J^og(^J exp I (g? + m^{t)) dW^ - ^ (G? + m^(i))' dij 



V^dWr - f Vt (G," + m^Q{t))dt \ ) d7Q ® 71bP MQ. + 0{s) 



Integrating with respect to 71 gP and using the independence of G and V and the fact that V is 
centered, we obtain the desired result. 

This lemma ensures that any minimum of H is equivalent to P. Indeed, if 5" > 0, then the result 
of lemma [4] implies that H{Qs) — H{Q) ^ s51og(s) which is strictly negative for small s, contradicting 
the fact that we assumed that Q minimized H. We therefore necessarily have 5 — P{B) — 0, that is 

Let us now characterize the minima of H . To this purpose, we use a variational formulation to show 
that any minimum of H satisfies equation (15), and start by proving the following technical result: 

Lemma 5 Let <P be a positive and bounded measurable function on C such that J <PdQ = 1, and denote 
ij^^-l and Qs{<P) = Tq^Q. We have: 

- liQsmP) - I{Q\P) + 5 / tf'log §dQ + 0{s^) 

- FiQsm = r(Q)+.s| / (log /exp { I^iG? + m^it))dW," - i (Gf + m2^it)fdt}d^Q + Yt) d^Q- 

J YT{y)dQ{y) + Cq{<P)^ + 0{s^) whereYx is an adapted process with finite variation and Cq is a 
bounded function. 

Proof The first point is simply proved as follows: 

Noting that = I + sip + 0{s'^) and / ipdQ = 0, we readily obtain the desired result. 

The second point is slightly more delicate but based on the same argument as outlined in the proof 
of lemma |4[ We introduce two independent centered Gaussian processes G and V with covariances Kq 

and K^tQ respectively and build the Gaussian process '^J^j^^ that has the covariance given by KQ^(^^y 
We then express r{Qs{^)) as: 

f f f (sr^ G? + ^/sV,°' m9y(t) + sm%r.{t) 

--2L (^7rTT^+ ° 1 + ., ° ) "'h' 
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and a series expansion ensures that: 



VT+~s 



1 + s 



l + s 



exp{X:£ 

= exp{^^ {Gf + m^{t))dW,"-ll^ {Gf + m^{t))' dt} 

Crp rp \ ^ 

^ V-dW,"-J^ V,"{Gf+m^{t))dt\ + 

{V-r + 2(G? + m^m-^ + m^Qit) - (f))) dt 

( £ '^"dW,- - j'^ y," (G? + m^(f)) dt^ ( V,^dW^ - j'^ (g7 + (f)) dt 



dt\ 



Integrating with respect to 7<fQ and injecting this expansion in the expression of F yields the expression: 

r(g,(<z>)) = r(Q) + s| /fiogy exp{^y^(G," + m^)dW^« 



with 



Xt{x, ^) = ^ y d7*Q I \ V.^dW^ - £ V," (G« + m§(i)) dt 
{t)+m%Qit)yW^-\j^ 



+ 

and 
rf7^(w) = 



4— §^ 



exp {Ea Jo" + ™Q W) dW-{x) - \ (G«(a;) + m^(t))' di} 



It is easy to see that m%Q = m%Q — rriQ, so that: 



XT(x,f)=^||i^ V-dW^-j^ y,"(G,"+m^(t))dtj +y (-^+mjQ(0)cil^r 



or, 



C 
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where 



YT{x,y) = J2^\ r S^,{y]_^JdW- 



and 



K$Qit,t)dt- m^Qit)m'^{t)dt 
Let Yxiy) — f J Yt{x ,y)d^^ dQ{x) . As Yrix^y) > 0, Fubini Theorem ensures that: 



XT{x,^)dQ{x) = J YTiy)d^Q{y) + J YT{y)dQiy) +Mq 



Cq{x,1>) 



As Va e {1 • • -MlV/i, < < we conclude by stating that Cq{x,<P) is an Q-integrable 

process, which ends the proof. 



We are now able to prove that any minimum satisfies equation (15 1. A necessary condition for Q 
to minimize H is 

\iia-{H{Q,{^))~ H{Q))>Q. (16) 



s-!-0 S 



But Lemma [5] implies that, for every such that J <l>dQ = 1 , 

H{Q,{1>)) - H{Q) ^ I log III - log I exp { ^ + m§(t)dW'f 

(G?+m5(t))'rfi}d7Q) ->T|rf'Z'g-s|y"yTdQ + CQ(<Z>)|+0(st) (17) 



Let 



y 1 1 



We can rewrite (17) as 



H{Qs{^)) - H{Q) = s y" ZTifdQ ~ ^{ / ^^"'^ + ^qI*^)} + ^(s^). 

Lets show that for any bounded measurable function 'P with j ^dQ — 0, we have j 'FZxdQ — and, 
as a consequence, we can find a constant cg such that Z-p = cg almost surely under Q, and thus P- 
almost surely. Indeed, if not, we can find If such that / Zr'^dQ ^ and / ^dQ = 0. Now let !fc = cW. 
Choosing c = —d sign( J" Zr'I'dQ) with d > large enough, it is clear that 'Fc satisfies the condition of 
Lemma [Sj and moreover: 

lim - {H{Q,{$,)) - H{Q)) - c /" Zr^dQ - / yT^Q - Cq{$,) 

s^Q S J J 

< -d| J Zr^dQl ~ J YrdQ + cg 



which is strictly negative for d big enough. Hence, we find a contradiction with condition (16 1, so 
that P almost surely we have the equality Zt — cq . But must be a (C, {Ft)t<Ti^TTP) 
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local martingale. Since ( / exp | Gf + m° - i (cf + m° (t)) ^dt|d7Q j is a lo- 



t<T 



cal martingale and (Yt)«T a process with finite variation, we have by uniqueness of semimartingale 
decomposition, we conclude that: 



^ = /"exp <j ^ r + m'^{t)dW," - \ f"^ {Gf + m'^Q{t)fdt ) d^Q. 



3.3 Existence, Uniqueness and precise characterization of the limit 



As a consequence of the form of the density obtained from equation (151 and the fact that the law of the 
uncoupled process P is Gaussian, it is easy to show that when considering Gaussian initial conditions, 
any possible minimum of if is a Gaussian process. Indeed, the characterization of the minima readily 
implies (as a simple application of Girsanov theorem) that the possible limits of the network equations 
are the law of the solutions of the implicit equation: 



1 



dX^ = ( --^Xr + Ut"" ] dt + KdW," 



(18) 



where the processes (W") are independent Brownian motions and the processes U^''^ are Gaussian 
processes with mean 

M 

m"(i) = ^Jc<^E[5„^(X7_,^J] 

7=1 

and covariance C"'^{t, s) = if a 7^ 7 and 

M 
7=1 



where 



The relation given by ( 15 ) provides a self-consistent equation on the moments of the Gaussian, which 
is given in the following: 

Theorem 7 Considering that the initial conditions are Gaussian, then possible minimum Q = (Q", a = 
1 • • • A/) of the good rate function is Gaussian. Denoting by n"{t) the mean of Q" and by G"'^ {t, s) = 
]E[X"Xf ] their covariance, we have: 



1 

"(i) = — — — /i"(t) + ^ ^ Ja^fa^{lJ-'^{t — Tct-y), G°'°'{t — Tq,^, t — Taj)) 



(19) 



7=1 



where faj{fJ,,v) = Sa-y (x) "^ ' — "^^^ covariance is equal to zero when (3 ^ a and: 



C"(t,s) ==e-(*+^)/''° C"(O,O)+^(cxp2(tAs)/0„-l)+^a2 / / e("+^')/'^"Z\;:},(u-T,^, i;-r„^)dwdi; 



M 



7=1 



-'0 



(20) 

where A'^^{u,v) = E Sa'y{X2)Saj{X2) is a nonlinear function of fi'^{u), fi^^v), G'^^ {u,v), G^^{u,u) 
and G'^^ {v, v). 

Moreover, there exists a unique solutions to these self consistent equations (19) and (20). 
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The above theorem hence characterizes univocally the hmits of the network equations considered. 



The proof of this proposition was done in |12j starting from equations ( 18 ) which were introduced 
using a heuristic argument. 

Note also that if the initial condition is not Gaussian, the solutions are not Gaussian. However, 
as time goes by, solutions get exponentially fast attracted to the Gaussian solutions described in 
theorem [?) That description hence provides a handy procedure to analyze the solutions of the mean- 
field equations and their dynamics as a function of the parameters. In particular, we observe that the 
levels of heterogeneity, (cFaj), appear as parameters of the equations. The moment equations provided 
above hence allow analyzing the qualitative effects of heterogeneity on the behavior of the network. 

If one is interested in the possible solutions of the mean-field equations for non-Gaussian initial 
conditions, or when the intrinsic dynamics of the system is not linear (i.e. P is not a Gaussian measure). 



existence and uniqueness of solutions to the implicit equations ( 15 ) also hold, but the demonstration 
of this property is more involved. The basic idea of the proof is based on a contraction principle, as 
usually done for proving existence and uniqueness of solutions. It is possible to show that one has a 
contraction in the Vaserstein distance. The steps of the proof are given in a different context in [31 
Section 5.2], and is not given here since we are not dealing with these more general cases. 

3.4 Convergence of the process 

We are now in a position to prove theorem [T] 

Proof (Theorem^ Indeed, for S a strictly positive real number and B(Q,S) the open ball of radius 5 
centered in Q for the Vaserstein distance. We prove that (5"(/i„ ^ B{Q, S)) tends to zero as n goes to 
infinity. Indeed, for a compact defined in theorem (6] we have for any e > 0: 

Q"(An ^ BiQ, S))<e + Q'\fLn e BiQ, Sy n K,). 

The set B{Q, (1 is a compact, and theorem [5] now ensures that 

lim sup - logg"(A„ e B{Q,6Y D K,) < - inf H 

n^oo n B{Q,synK, 

and eventually, theorem [4] ensures that the righthand side of the inequality is strictly negative, which 
implies that 

lim Q"(/i„ ^ B(Q,<5)) <£, 

n— ^oo 

that is: 

lim Q"{finiB{Q,5))=0. 

Based on this result, we can further conclude on the following: 

Theorem 8 The system enjoys the propagation of chaos property. In other terms, Q" = ]Ej[(Q"( J)] is 
Q-chaotic, i.e. for any bounded continuous functions (/i, • • • , fm) and any neuron indexes (/ci, • • • , km), 
we have: 

/m m „ 

\{f,{x^^)dQ''{x) = n / f,{x)dQ^^^\x) 

This is a direct consequence theorem [T] thanks to a result due to Alain-Sol Sznitman, see |2H Lemma 
3-1]. 

All these results can be readily confirmed by numerical simulations of the network equations. 
Considering for instance a two-populations network with parameters given in section [T4{ we simulated 
a network of 12 000 neurons (6 000 in each population) and considered the distribution of the values 
of the membrane potentials as a statistical sample. The empirical distribution, superimposed with 
the theoretical Gaussian distribution, is plotted in figure Fig. [l] and shows a very clear fit, which 
we confirmed using the Kolmogorov-Smirnov test. For each population, the Kolmogorov-Smirnov test 
comparing the sample obtained by numerical simulations with the predicted Gaussian distribution 
ensures that the sample has indeed the Gaussian distribution, with a p- value equal to 1. Moreover, we 
used a chi-square test of independence which validates the independence between the two populations 
and this independence test was validated with a p-value of 0.87. 
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Fig. 1 Empirical distribution (colored histogram with dotted li nes) and theoretical Gaussian distributions 
for a 2 populations neuronal ne1;work (parameters given in section [Xl] with common heterogeneity parameter 
a — 1 and noise A = 0.5). 

4 Dynamics and phase transitions 

In this section, we analyze the obtained limit equations to analyze the quahtative macroscopic be- 
havior of networks, with a particular focus on the effect of the disorder parameters. The first step of 
this analysis consists in confronting our results with the seminal study of Sompolinsky, Crisanti and 
Sommers (SCS) [5D] dealing with one population model with centered coefficients, centered sigmoidal 
functions S and no delays. We will then discuss the persistence of the phase transition they identified 
when the assumptions on the parameters (non-centered synaptic weights or sigmoids, delays, multiple 
populations. . . ) are relaxed. A particularly important phenomenon in neuroscience essentially absent 
of their initial study, synchronized oscillations, will be an important focus of the present section. 

Let us eventually note that the results of SCS hold for zero-noise limits (A^ ~ 0). We will also 
sometimes consider this limit equations, even if rigorously, the proofs of section |3] hold for non-trivial 
noise. These non-noisy regimes correspond to limits of the mean-field equations where — ^ 0, and 
correspond to a sort of viscosity solution of the system: all the properties of convergence, existence 
and uniqueness of solution hold for arbitrarily small A^ and provide solutions that have a limit when 
Aa ^ 0. 



4.1 The generalized Somplinsky-Crisanti-Sommers Equations 

In their article, Sompolinsky, Crisanti and Sommers (SCS) introduce a set of equations governing the 
dynamics of covariance of possible stationary solutions to the mean-field equations. These equations 
are used to analyze the dynamics of the limit process and in particular to show a striking transition 
between stationary and chaotic solutions. We derive here a generalized equation of the type of the SCS 
equations in our framework with multiple populations and delays, and use these equations to explore 
the boundaries of the SCS phase transition when considering different models. 

Proposition 4 Possible stationary solutions are Gaussian with mean fl" and covariance C°'{t) = 
(jaa^-f- _|_ j^^^ ^ > These two variables satisfy the system of equations: 




(21) 
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Remark 2 Note that the above equations do not constitute a dynamical system, but rather correspond 
to imphcit equations. In particular, an important difficulty is the choice of the initial condition C"(0) 
which corresponds to the variance of the stationary solution, which is obviously unknown. This quantity 
parametrizes both the equation on the first moment and the form of the term Zi"^ on the second 
moment equations. 

Proof The equation on the mean p," is a simple rewriting of equation ([T]) under stationarity condition. 
The equation on the stationary covariance requires more care. For arbitrary time t, denoting X" the 
solution of the mean-field equation with for all a, \a = 0, we have, using equation (|9]): 

c'^io = ^mx'^it + - + c))(^" w - 

= -^-r^+E[X"(i);7"'^(t + C)]. (22) 

The second term is not easy to characterize. The method used by Sompolinsky and collaborators to deal 
with this term is to derive a second time with respect to C. However, the differential of U" is unknown. 
Fortunately, we can express this term as a function of (5"(C) = ]E[X"(i + This function is 



way easier to handle since using the differential equation ( 18 ) and differentiating this expression with 
respect to f , one obtains: 

" l3=l 

where we denoted with a slight abuse of notations Z\°^'^(C) the common value of A'^^^it + C,, t) for any 
t > Q using the assumed stationarity of the solution. 

In order to relate the second term of the righthand side of ( 22 ) with (5" , we compute C{(^ + 
expressing it the differential with respect to ^ of ]E[X"(t + C + S,/2)X°-{t — ^/2)]. In this computation, 
most of the terms cancel out and we obtain the simple expression at ^ = 0: 

2C'"(C) E[X"(t)C/"^^(t + C)] - (5"(C)- 
Plugging this expression into ( p2| we obtain: 

fa 

Differentiating this expression with respect to C and reinjecting the latter equation in the obtained 
expression, we get: 



[j=i 

M 

-Y^alpA-^iO- (23) 

l3 = l 



This equation is very similar to the original SCS equation. As they remarked, this equation does not 
characterize the process. Indeed, we know that C"(0) — using the fact that the covariance is even, 
but the initial condition C"(0) is not fixed: it is the asymptotic stationary variance of the process, 
when it exists, and this initial condition is a parameter of both the stationary mean equation and 
stationary covariance equation. Let us emphasize also the that it does not involve the average of the 
synaptic coefficients Jap, and that the function A"'^{Q is a nonlinear function of C^{C) and C^(0) 
which can be written as: 

A'^f'{o = nsa0{v^{O)Sa0{vf'{m 



R2 
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where Dx and Dy are the probabihty measure of standard Gaussian random variables {Dx — e^^ /\/2TTdx 
and similarly for y). The linear term has a positive sign, hallmark of diverging systems. And indeed, 
the solution can be formally written as: 

C"(C) = C"(0)cosh(^)-^^( / e^Zi^CK- / e-'-^ApiOddi. 

In that equation, the term in hyperbolic cosine diverges very fast, and the nonlinear term can overcome 
the divergence. When one does not consider the precise initial condition corresponding to stationary 
solutions, the solutions of the second order ODE in C" diverge very fast, as remarked by Sompolinsky 
and colleagues in |20j, and therefore one needs to analyze the stability of the possible solutions, which is 
relatively complex to perform. Numerical simulations of the system are very intricate as well, because 
of the time-consuming calculation of of the nonlinear function A and because of the possible divergence 
of the solutions. 

However, Sompolinsky and collaborators show very elegantly an important phase transition taking 
place in this system, analyzing the shape of the potential together with a stability analysis of the 
solutions. We revisit their results in our more general framework, first in one population systems, and 
then in higher dimensional systems, and particularly focus on the effects of delays, non-zero mean 
connectivity and non-centered sigmoids. 



4.2 One population networks 



The heterogeneity level appears as a parameter in ( |23| . In their one-population setting, equation ( 23 1 
can be written the equation of the position of a particle submitted to a force deriving from a potential 
^1 (the label 1 denotes the number of populations) which is equal to — ^C^-|-cr^^ where ip is a primitive 
of A considered as a function of C . The shape of the potential showing a transition from convex to 
double- well as a is increased allowed the authors to conclude on a phase transition between a stationary 



solution where ( 23 1 has a unique equilibrium equal to zero and a chaotic regime where the covariance 



is non-zero. The value of the noise at this transition corresponds to ct5'(0) — l/r. 
4-.2.1 Non-delayed networks with non-centered synapses 

Let us start by a one-population network with no delays and J 7^ 0. In that case, it is easy to see 
that fixed point with mean /i = is stable if and only if J §^(0, C(0)) < 1. When letting tr fixed and 

increasing J, we can see that at J = ^(0,C(0))^^, the system undergoes a pitchfork bifurcation, 

and two new equilibria fi^ > and fi~ < appear, which are stable. For these equilibria, the null 
covariance is no more a solution to the equations, and we observe a stationary behavior of neurons with 
a non-zero standard deviation, i.e. a dispersion of the individual trajectories, that remain stationary. 
At these points, the system undergoes also a phase transition from stationary to chaotic activity when 
the heterogeneity coefficient crosses the critical value a = C(0)). This equation is an implicit 

equation since the righthand side depends on a through the stationary variance of the process. Let us 
denote by r{a) the stationary standard deviation C(0). It is clear that r{<T) is an increasing function of 
cr, and to fix ideas, let us consider that the sigmoid used is an erf function S{x) — erf(g x) — J^^ e~^'/^. 
Then it is easy to show using a change of variables (see [25]) that 

/(x,a)^erf( , ^ ) 



and therefore the pitchfork bifurcation arises along the parameter curve — y p" + ^^'^ the 

phase transition from stationary to chaotic behavior along the curve —1-1- aS'{ii^) = 0. Since the 
differential of S takes its maximum at and decreases to zero at ±00, the value of a corresponding 
to the secondary phase transition to chaos is an increasing function of a. Moreover, in that case, the 
chaotic activity will be no more centered around zero but around the new fixed point is^. Eventually, 
it is interesting to note that for J smaller than the value corresponding to the pitchfork bifurcation, 
the stationary covariance r{(7) precisely equal to zero. A hand-drawn bifurcation diagram reflecting 
this behavior, together with simulations of the trajectories, is plotted in figure Fig. [2] 
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Fig. 2 Behavior of a one-population system with non-centered synaptic coefficients. Center: Bifurcation dia- 
gram (hand-drawn) segmented into four regions: two regions of stationary behavior (yellow: centered at zero 
and green: centered on /i*) and two chaotic regions (pink: centered around zero and orange: centered around 
/X*). The boundaries of these regions are: a pitchfork bifurcation (blue curve) separating the stationary or 
chaotic regions centered on to the ones centered on /x*, and a generalized SCS phase transition (red curve) 
separating the stationary and chaotic regimes. The subfigures (a)-(e) show the time course of 30 arbitrarily 
chosen neurons in the network corresponding to the points a-e of the diagram: r = 1, ^'(O) — 1, (a): J — 0.5, 
a = 0.5, (b): J = 0.5, a = 1.5, (c): J = 1.5, a = 1.5, (d): J = 1.5, a = 1.7, (e): J = 1.5, a = 2. 

4-2.2 Delay-induced oscillations 

We now consider a one-population network with delays. Without loss of generality, we consider that 
the time constant is equal to 1. The solutions of the mean- field equations with no heterogeneity are 
Gaussian processes whose moments reduce to a dynamical system: 

fi = + Jf{^{t-T),v{t-T)) 

and hence the variance converges towards To fix ideas, we consider S{x) = eTf{gx), so that 

f{x,a) = erf( j^^^ ^ ). Since for any v, f{0,v) = 0, the null mean is a stationary solution of the 

equation. Its stability depends on the roots of the characteristic equation (or dispersion relationship): 
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If all characteristic roots have negative real part, the fixed point /i = is stable. As a function of the 
parameters of the system, characteristic roots can cross the imaginary axis and yield a destabilization 
of the fixed point. Turing-Hopf instabilities arise when there exists purely imaginary characteristic 
roots ^ = iw. In that case, we obtain the following equivalent system: 



-l + J 



■ cos{ujt) = 
= sin(a;r) 



which has real solutions only for J- 



arise when the parameters satisfy the relationship 



> 1. It is then easy to show that Turing-Hopf bifurcations 



arccos 



V 



Jg 



and these correspond to characteristic roots w — 



1. These result into oscillations of the 



solutions at a pulsation equal by oj. 

Let us now return to the case of random coefficients with variance a and no additive noise A = 0. 
The mean of the Gaussian solution satisfy the same equation as the one studied above with A = r(a)^ 
and as noted in the previous section, the stationary covariance is an increasing function of a. For 
J > 5^(0,0), the fixed point is unstable, and the covariance is non-zero. This implies that for 
sufficiently large values of the delay, the network displays oscillations. Thanks to the propagation of 
chaos property, all neurons have the same distribution, which is a Gaussian with oscillatory mean, and 
hence the network displays phase-locked oscillations. Eventually, as noise is increased beyond a critical 
value, a SCS phase transition occurs and the system no more displays phase locked oscillations but 
asynchronous chaotic activity. This is illustrated in figure Fig. [3] 



4.3 Multi-population networks 

In this section, we analyze the dynamics of randomly coupled neuronal networks in the case of the 
deterministic coupling of several original SCS networks, before turning the analysis of the dynamics of a 
more biologically plausible neuronal network composed of an excitatory and an inhibitory population. 
As demonstrated by Sompolinsky and coworkers in [20| . the study of the stationary states using 



equations (21) is very useful to analyze the dynamics of their networks. Unfortunately, this method 
does not persists in higher dimensions, since the equation does not necessarily derive from a potential. 
Indeed, in order for the equation to derive from a potential <?m : IR^^ i— >■ R in dimension M greater 
than 1, we need that for any a € {1, ... , M}: 



/3=1 

The only case where this is possible is the case where cr^^ = for any a ^ /3. Indeed, shall the above 
relationship be true, the equality gQ^gQ^ = gc^dC'^ directly yields cr^^|^ = (t^q,|^. The lefthand 
side is a function of C" only and the righthand side a function of only, they for a ^ f3 these 
functions are necessarily constant. For regular functions S, this necessitates to have CTq^ = apa = 0. 
This is precisely the case of deterministic lateral connections between randomly coupled networks, 
which will now study. 
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(a) Turing- Hopf bifurcation (b) r = 0.1, a = 0.5 

curve 
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(c) r = 0.5, a = 0.5 (d) r = 0.5, a = 1 

Fig. 3 One population delayed system, r = 1, S{x) = eTf{gx), g — 3 and J — —2. (a) represents the curve of 
Turing-Hopf bifurcations in the plane (t. A) obtained analytically, (b): r small: no oscillations. As the delays are 
increased, a Turing Hopf bifurcation occurs and oscillations appear (b), which disappear when the heterogeneity 
a is increased beyond a critical value in favor a chaotic activity (c). 



4-.3.1 Deterministic lateral coupling of SCS networks 

In this section we analyze the coupling of different SCS networks, called lateral coupling, with deter- 
ministic coefficients. The only randomness in the models is included in the random synaptic coefficients 



between neurons belonging to the same population. In that particular case, equation ( 23 1 derives from 
the potential <Pm{C'^ ■ ■ ■ C^) = EoLi ^i(C"), and in that case the analysis driven by Sompolinsky 
and collaborators can be adapted to the multi-dimensional case. Since the potential is now the sum 
of the individual potentials at each population, we observe a strange phenomenon of localization of 
chaos in the populations that display a large heterogeneity (namely, in our notations, when the SCS 
condition cr^a^aai^)'''a > 1 is satisfied). Only the populations that individually would be in a chaotic 
state are in a chaotic state, and the other populations converge to zero with a Dirac delta covariance 
at zero, and the input received by such populations from chaotic populations do not perturb this state. 
Let us for instance illustrate this phenomenon on a two-populations network with parameters: 

Each population receives input from the neurons of the other population, with a constant synaptic 
weight equal to Jq^, and the intra-population synaptic weights are noisy. 

Further analysis of this networks as a function of the coupling reveals a similar phenomenon as 



the one described in the one-population network of section 4.2.1 Indeed, as the strength of the lateral 
coupling Ji2 and J21 are increased, additional stationary solutions with non-zero covariance appear. 
Let us for instance denote by /i* the mean of one of these stationary solutions. Following SCS analysis. 
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(a) Shape of the potential (b) Trajectories 

Fig. 4 Dynamics for a 2-populations network with Hamiltonian dynamics (no variance on the cross-population 
synaptic weights): ri = r2 = 1, (Ti = 3 > 1, (T2 = 0.5 < 1, J12 = J21 = 3. The potential shows a double-well 
shape, corresponding to a chaotic state on population 1 and a stationary state on population 2. Simulation of 
a 4 000 neurons network illustrate this phenomenon (right): blue (resp. red): 30 arbitrarily chosen trajectories 
population from 1 (resp. 2). 



we are ensured that the behavior of the trajectories of neurons in population a around /Lt* is stationary 
as long as ctq < ^ s\p' ) chaotic otherwise, and this independently of the behavior of the other 
population. This phenomenon is illustrated in figure Fig. [5j 
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(a) Stationary States (b) Localized Chaos 

Fig. 5 Stationary and partially chaotic solutions of a two populations network with deterministic lateral 
around non-zero fixed points. J12 = J21 = ^, (J2 = 0.5, (left): a\ = 2: both populations display a stationary 
behavior, (right): o\ — 5: only population f is chaotic and the irregularity is not transmitted to population 2. 



4.4 Heterogeneity-induced oscillations in two-populations networks 

We eventually discuss the effect of heterogeneities in a more biologically plausible neuronal network 
including one excitatory and one inhibitory population, with strictly positive sigmoidal transforms 
(since these functions model the input to firing-rate transformation), that tend to zero at —00 and 
to 1 at 00. This system was analyzed in [24]. Considering M = 2 populations, all sigmoids equal to 
erf{gx) — ^"^ / \/2tt dy (yielding fap{x,v) = eYi{gx/ \Jl -+ g'^v)), all time constants da = 1, and 
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(a) Bifurcation Diagram 






(b) a = 0.9 



(c) a = 1.6 



(d) a = 3.5 



Fig. 6 Heterogeneity-induced o scill ations in a two populations excitatory-inhibitory network, (a) Bifurcation 
diagram of the system of ODEs (19 1 as a function of a presumably constant common value r{a). LP: saddle- 
node bifurcation, H: Hopf bifurcation, Sh; Saddle-homoclinic bifurcation, blue: fixed points (solid: stable, 
dashed: unstable), magenta: limit cycles, (b)-(d): simulations of the heterogeneous network with 2 000 neurons 
per population, for increasing values of the heterogeneity parameter a displays a transition from stationary 
to periodic phase-locked behaviors, and then to chaotic behavior, blue (resp. red): 30 arbitrary trajectories of 
population 1 (resp. 2), cyan (resp. magenta): average on all neurons of population 1 (resp. 2). 



the connectivity matrix, inspired from the seminal article of Wilson and Cowan 

-_/l5-12 
^ 1^16 -5 

we showed that the system presents phase transitions as a function of the heterogeneity parameter, be- 
tween stationary distributions to periodic oscillations (see figure Fig. [6]): considering all equal and 
denoting a the common value, we observe that for small heterogeneity parameter u, the network con- 
verges towards a stationary solution with non-zero mean. For intermediate values of the heterogeneity, 
phase-locked perfectly periodic behaviors appear at the network level, that disappear, as heterogeneity 
is further increased, through a SCS phase transition yielding chaotic activity. 

This phenomenon relates the level of heterogeneity to the presence of synchronized oscillations in 
networks, an essential phenomenon, as we discuss in the conclusion. 



5 Discussion 



In this manuscript, we analyzed randomly coupled neuronal networks and derived their limit as the 
number of neurons tends to infinity. To this purpose, we showed that the system satisfied a large 
deviation principle and exhibited the related good rate function. This approach generalized the work 
of Gerard Ben Arous and Alice Guionnet [31113] developed for spin glasses in three main directions: 

(i) the synaptic weights are not centered, introducing additional, deterministic terms in the couphng, 

(ii) interactions are delayed, which projects the problem into infinite dimensions, and (iii) the system 
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is composed of several populations, which was handled showing that empirical measures on each pop- 
ulations simultaneously satisfy a large deviation principle. The proof is made on a particular model 
very popular in physics and neurosciences, the Wilson and Cowan system, which is close of the famous 
Hopfield model, but as mentioned, can be easily generalized to nonlinear neuron models. Indeed, most 
of the proofs deal a quantity which is related to the density of the coupled network with respect to 
the uncoupled dynamics, and this quantity is independent of the dynamics of individual cells. Our 
approach can be also readily extended to networks with distributed delays. Eventually, let us note that 
this result provides large-deviations estimates on the convergence of deterministically coupled networks 
as studied in |25j . 

From the neuroscience viewpoint, this approach has the interest to justify an equation which has 
been widely used in the analysis of large-scale networks, and introduced in the seminal paper of Som- 
polinsky and collaborators |20|. Moreover, our setting substantially extends their result by taking into 
account important features present in cortical networks: interconnection delays, multiple populations 
with non-zero average synaptic connection. All these refinements allowed going deeper into the un- 
derstanding of the dynamics of neuronal networks. In particular, we showed that delays can induce 
oscillations in one-populations networks modulated by the level of heterogeneity, and that non-zero 
average connectivity yields non-trivial dynamics that were not present in the original SCS model. 
Moreover, we showed that networks with multiple populations can show relative counter-intuitive phe- 
nomena such as the localization of chaos: a few populations can have a chaotic behavior which is 
not transmitted to the other populations, whatever the connection strength. Another phenomenon we 
illustrated was the apparition of heterogeneity-induced oscillations, phenomenon first presented in a 
very recent article j24|- As discussed in that article, the latter phenomenon can be related to experi- 
mental studies that showed that the cortex of developing rats subject to absence seizures (abnormal 
synchronization of some cortical areas) was characterized by the same mean synaptic connectivity, but 
increased dispersion of the synaptic weights [2]. We further showed here that such oscillations were 
facilitated by the presence of delays. Let us eventually underline that the particular form of our system 
is not essential in the apparition of such phenomena, and in j24l . it is shown that the transition to 
synchronized activity as a function of heterogeneity levels persists for realistic neuronal networks made 
of excitable cells, the Fitzhugh-Nagumo model. 

An important observations is that in all the examples treated, the SCS phase transition to chaos 
is present as the heterogeneity is increased. This phenomenon seems relatively universal in this kind 
of randomly coupled neuronal networks. It was very recently related to the topological complexity of 
the underlying vector field in [27j in the original SCS framework, and we conjecture that the same 
phenomenon occurs in our more complex settings. 

The analysis of the present manuscript underlines the fact that the structure of connectivity maps 
is essential to the function of the networks, and illustrated the fact that averaging effects do not cancel 
the structure into populations and allow serving functions such as oscillations. However, actual brain 
connectivity maps are not recurrent, and can display different topologies, with different computational 
capabilities. The extension of our methods to such networks is an active part of our future research. 
Moreover, our analysis did not take into account the plasticity mechanisms, resulting in the slow 
evolution of the synaptic weights as a function of the activity of neurons, which tends to correlate the 
synaptic weights to the voltage variables. Techniques to rigorously address the dynamics of neuronal 
networks with non-recurrent connectivity, with specific topologies, or with correlated synaptic weights, 
are deep questions that are still largely left unexplored, and we can expect that a wide range of novel 
phenomena will arise from the analysis of such networks. 



A Proofs 

A.l Proof of Lemma [T] 

This appendix is the proof of lemmafTlconcerned with the properties of the functions used in our large deviation 
principle in a discrete time framework. 

Proof Before proceeding to the proof of the lemma, let us start by showing an inequality whi ch will be very 
useful in several steps of the demonstration. We recall the following well known formula (see |10l Lemma 3.2.13] 
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for instance); 

I{fi\P) = sup (^J <Pdfi - log J exp$dP ; <P G Ct{C) 
so that, for any bounded measurable function ^ on C, we have: 



J $d^^< I {^i\P)+ log J exp$dP (24) 



The result hold for positive measurable functions by the monotone convergence theorem. 
Proof of Lemma [Tj(i). 

Lets prove the lipschitzianity result for _r"'*. We have: 
log 1+ -, — T;^. TTVl = A (m) - A ■ («^) + o / (("^m) - I™--') )(* m 



/exp(-lEtoG,f (i^+i-tO)d7AV ^11^^^^ ^ 2 

- ^^"^ { 2^ } I y ™^ ^ " 2 /o ^'^^^ '^^^'^'^ ~ •* I 

Let 5 be a probability measure on C x C with marginals ^ and v, and let 75 be the law of a bidimensional 
centered gaussian process {G",G°') with covariance K^: 

M _2 

A"; 



( ,N _ V- crQ7 //'S'<:«7(2;I-r„^)'S'c«7(a;7-rc^)c'C(a:,y) /S'aT.(a;]_^^^)S'„^(y7_^^^)dC(a;,y)\ 



Then, 



n (/ r(G,%, +eG^)'dtd7e 



1 

< 

- 2 



by Cauchy-Schwarz inequality. Then, using the covariance of {G°',G°') under 7^, we find: 

|rr^^(M) - ''=(0 + y {{m-f - (m^)^)(t('=')dt| (26) 



< I^VTexpj^} ^max^^ | J j^^ \S^,{xU^^) - S^,{y2_^^_^f dtd^{x., 

(27) 



7 = 1 



Moreover, we have: 



|m^(t«)'-m^(tW)'dt| = j |(m^(t(''))-m^(t(*='))(m^(t«)+m^(t(''-)))|dt 

j |m^(f(*))-m^(t«)|dt 



< 2 — 



32 



But 

fT I 1 - r 

I dt 



[ - rnZ){t)\dt = [ \^'S^ J^^ ( Sa-i{xl_^)d{ii. - v){x)\ 

Jo Jo ' -^a J ' 

-^|Ja7l / \ Sc^{x'^t_^^_^)d{fi-u){x)\dt 
J Jo ' J ' 



< 

7=1 



<3-El'^"^l / / \Sc.,ixt.^_^)-S^^{y]_^)\dtdax,y) 



7 = 1 

^ J a 

< - — max 

Xa 7=1---A/ 







T 







by Cauchy-Schwarz inequality. 
Consequently: 



'XI 'XI '^UA^ JS=i- M 

As the S'a7 are Ks Lipschitz, we have: 

|^.,.(^)_^.,.(^)^ fc|^^^p|fc^|^^^(^^^)^ (29) 

so that J^"'* is Lipschitz for the Vaserstein distance. Using the triangle inequality, the result holds for J^. 



Proof of Lemmc(T|(ii): 

Let 



F^{x) = log 1 1 e^p{£ (G,(.)H + m^(t('='))' ■ dWi(x) \\g,w{oj) + m^{t^''^)f dt]d^. 



This function is a.s. finite but not bounded, let us hence define for A G IR 



F^(x) = log II A A exp { ^ (G,(., (a;) + m^(t(*'))' ■ dW,(x) - ^ | ||g,(.) (o;) + m^(t('=')||'dt}d7. 
By the monotone convergence theorem and using equation (24 1, we have for any a > 1: 

a j F^{x)dii{x) < J(At|P) + log| J cxp aF^{x)dP{x)] 
By Jensen inequality and Fubini theorem, 

J exp{aF^{x))dPix)< J H J exp {a|^ (G^,.) + m;:(t<'=^))dW^f (a;)}dPc«(a;) 

exp|-^y ||Gj(fc) (oj) + m^(t'''')|| dt^dj^. 

But, as W" is a Pa-Brownian motion, 

y exp I a (G7(.) + m° (t )) dW^f (x) } dP^ (x) = exp | ^ ^ ^ (G^',., + 'dt} 

so that, 

a |F^(x)dM^) </(/i|P) + log||exp{"^y^||G,(.,(a;) + m^(t(*')||'dt}d7^|. 
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Letting a = 1 proves that r'' < I{\P). 
Proof of Lemmc(T] (iii): 

As the components of G are independent under 7^, we only have to check that, for every 6 > 0, there exists a 
finite constant Cb such that 



exp 



b 
2 



. bCbkaT 
<exp— 



(30) 



It was proved in [3] Lemma A. 3(2)] in their particular framework that for every b verifying ^^§^ < 1, there 
exists a finite constant Cb such that: 



f b f'^ 



bcbkciT 



In our case, the covariance function is slightly different of that of [3] , but the proof and result remain unchanged 
and can be readily extended. 
Moreover, since we have 



(G? + m^(s))' < 2Gf + 2m2(s)^ < 2Gf + 2 



r 



we obtain the desired result with the following constant Cb ~ 2c2b + if^, under the condition ^''^2 < 1. 



Proof of Lemme(T|(iv) 

As above, lets prove the result for j-T^jf — -fT''^]- We have: 



\rlt(p)^r^'\^^)\ <\\j j [[j G,%,(dIVr-m^(tW)dt))'-( j G,%,(dH/r-m^(t(^-))di))'}d7^T,.dM| 

+ \\f j [j G,"(.,(dM/r - mZ{t^''^)dt)fd[^^T,u ~ 7kJ-0'^mI + \jf {mZ - m'^^){t^''^)dW^ di,\ 

Let ^ be a probability measure on C x C with marginals ^ and v, and let 7^ be the law of a bidimensional 
centered gaussian process {G",G°') with covariance K^. Let 

^^p(-UoG^,,,'dt) 

A^-\G°') = ^ . 

/exp(-|/,rG-,/dtjd75 

As in j3j lemma 3.4], we can show that: 



/ ■■ ^ 

:^{^^)-^,"'''{^^)\ <\ j j |a?''=(g") -/1?''=(g^-)|(| Gl,,{dw^ - mZ(t(''^)dt)f d^^d^ 

+ [j j ^t\G")[ j (G,"(., +£G'",(.))(dW^r -m^(tW)dt))'d7«dM 



+ 



A-'\G-){ ( / G,< 



.,(dVyr -m^(fW)dt))' - ( j G°(.,(dW^r -m^(fW)dt))'|d75dM| 



+ 1 / I / (m^ - m^^)(t(^-))dW^rfd/i ) ' (31) 



and 



yl°''=(G") =exp<^ -Tj 



"^''(m) mfit^''^)dt G-^^dt 
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Hence, we have by Jensen inequality, 
so that 

which eventually gives. 



+ 



Let h,m G L ([0;T], dt), with m bounded. By Cauchy-Schwarz and the relative entropy inequality ( |24[ ) 
(with <f (a;) — ^ Jp^ /itdW^t"(a;)^ ~ Af ^0, /g"^ htdtj under Pa, and besides is a positive and measurable function 
of C), we have the existence of a finite constant C such that, 

J (^j\t{dWt{x)~m{t)dt)^^dii{x)<2\^j j\tdWt°'y + [ J\tmtdty d^i^ 

< 2| (c{l + I{fi\P)) + mLrj ( hUt) 

<C'{l + I{ii\P))[j\ldt) (32) 

We can now bound the different terms in inequality (31 1. 

In fact, as ht = G"(fc) and mt = m'^(t^''^) verify the required condition, (32 1 gives the existence of ct, 

I [J^^^ G^w{dWrix)-m:{t"'^)dt)ydfi{x) < ct{1 + IMP)) I^G-^wfdt 

Hence, we can find a finite constant such that 

Bi < c^(l + J(/i|P)) ^max^^ ( J \Sa-,{xl,, - Sa-^iy},, dtdi{x,y)) ' 

Similarly, there exists a constant ct such that 

n {cT{l + I{i^\P)) j J\G-,,,+eG-,,.,fdtdf,y 



T 



<c'r{l + IMP)) ^max^^ ( J j^^ \Sa,{xl,,_^J - Sa,{yl,,_^J\^dtdax,y)) ' 
To bound Bs, we first use Cauchy-Schwarz inequality: 



^'-r^'PlS} n {/ /|^^G°<.,((l + e)dVyr-(m^(t«)+em2(t"=^))dt)pd75dM|' (33) 
I G^(., (m^(t«) - (*(*')) dtp < ( j'^^ G^^^.^dt) ( £ (m^ (*<*') - m^(tW))'dt) 



Remark that 



C 2 C 7 r 2 

I (m^(t«)-m^(t«)) dt = y^ (Ex^y dt 



7 = 1 

< ^ max /" f / 5'a-,(2;^,n )d(ti — iy)(x)^ dt 

72 

< — max , 



y [J \Scj{x'^^^k^_^^J - Scj{y'^^^^_^^_^)\d^{x,y)^ dt 
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So that 

f-T 



-'0 ' 



Moreover, ( 32 1 gives: 



Using the last two inequahties in ( 33 1 we have: 

/•T 



as J(|P) > 0. 

As of the last term, we have 



B4 < CT 



We have proved that there exist a constant ct such that 
\rltip) - < ct(1 + /(mI^')) ^max^^ (^^/ - y)dt) ' 

(34) 

And, therefore 

SO that using the triangle inequality 

I (m) - A' (m) I < Ct (1 + ^')) dT (m, J^) • 

. Proof of Lemmc(l|(v): 

For all Q G {l--~A.f}, let 

dQZ'^{x) = expr"'*(5^)dPa(a;) 

= y exp J^y^(G,"(.) +m^(t«))dM/r(a;) ~\ j^C^^,, + mZ{t'^''^) fdt^ dj.dP^ix) 

The equality between the two expression of Q"''' is easily obtained by gaussian calculus (see the proof of 
proposition pi . We deduce by the martingale property of this density that it is a probability measure on 
C{[^t,T],rY 
Remark that 



(-)=n 



dP ^ ' dPc 

a — 1 



a; 
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It follows that Qj: e X + (C), and 
dQt{x)^expr!:{S^)dP{x) 

= Jexp(^J^ (Gj,fc) +m,.(t<'='))' ■ dWt(T) - \\G^i,) + m,{t'-''^\^ d?j dj, dP{x) 

Lets now prove that Hj^ = J(.|QJ:). We will first show that /(QJ;|P) is finite. In fact, 



exp 



Which becomes, after some gaussian computations (see ^3, Lemma 5.15]): 

:exp|^^ff,%,(o-*^)dwr -Ij^ H^^,;\QZ''')dt 

where 



dPc 



= r (if*''=(t«,s«))_(dM/r -m^(s('=')ds) +m^(t«) 
Jo 

= E (Vy,':+, -H/,':-m^(tO(i^+i-tO)(i?i''(i''\ife)),.+"i°(i'") 

i=0,..,fc;t; + l<t 

Hence, according to Girsanov Theorem, there exists a Q"'*-brownian motion such that: 

w^ = B?+ riy;(.,(Q"'')ds 

Jo 

As ly is an afhnc function of B°' , it is, under QZ'^ , a gaussian variable with finite moments. In particular, 
I{QZ'''\Pa) is finite. But 

I{Ql\P) = / log (^{x))dQl{x) = Y.I l°g (^^{x-))dQZ'\x'^) = £ /(Q^''=|P.) 



so that /(QS|P) is finite. 
Now, let 



(^C(x) = inf|t> 0; ^*m^(s(*')dWr(x) 



> m 



be a sequence of stopping times for the brownian filtration (t(W",0 < s < T). As Vl^" is a P^-MB, we have 



lim 



oo almost surely under Pa- [Tm — mina=i...M 



' mgN 



defines a sequence stopping times for 



a(W",a = 1 • • • M, < s < T) which tends to infinity along with m P a.s. We define: 



Qi/,t 



(G^(.) +m,(s«))'- dW 



2 70 



G^(fc, +m,(s^'=') ds \\d-i^P 



tM = (/ { /* (•=^^<'=) (^) + m.(s*''))' ■ dW,(a;) 

--f\ 
'2 Jo I 



G^(fc) (a;) + in,^(s'''')|| ds [■d7^t (a;) ]dfi{x) 



as well as Q"'f and P"t* where 



1 r 



)Sa-,{xl_ )d^i{x) 
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is define on [0, t]^. 

These functions are clearly continuous in time on [0,r]. The result /(QJ^.t^atI^) < ^ obviously remains. 
By Jensen inequality, we have: 



1 /^ct,k 



> exp I 



2A2 



I exp I 



2X1 S ''''P 



{(x)AT 



m:{t'^''^)dwrix) 



so that, 



iVl 

-(x) > expj - ^ 



dP ^ 2X1 

a — 1 



Mm) 



We then apply the same proof as in [31 Appendix B] to find: 

G Mt{C), Hl^^^^-r = /(mIQ' r,„AT) 
Letting m to infinity, we conclude using the continuity of ^ and /(.|(5j;,t) on [0,r]. 

Proof of Lemme[l|(vi): In order to demonstrate that H*' is a good rate function, we need to show that it is 
lower semi-continuous and that it has compact level sets, i.e. {H^ < L} is a compact set for any L > 0. This 
is a direct consequence of points (i)-(iv) proved above. 



A. 2 Proof of Lemma [3] 

This appendix is concerned with the proof of lemma |3] ensuring an exponential bound that will be used to show 
a tightness result on the sequence of empirical laws. 



Proof Let 



B" — I exp 



|an(r(/i„) — r',/(/i„))|dQ" 



Writing the definitions of P and F^, we find: 



P n 





expj 


/,f (G, + m^„(t))'.dW?-lXf 


Gi +mji„(t) 




}] 




exp 


[/(f (G, + m.(t))'.dW?-i/,f 


Gt + m^(t) 




\] 





Let ^ be a probability measure on C x C with marginals jin and v. We then have: 



P n 





exp. 


/,f (G, + m^„(t))'.dW?-iXf 


Gt +m^„(t) 


^dt 






exp ■ 


/,f (G^ + m.(t))dWl-lXr|| 


GJ + m^(t) 


'^dt 


\] 





d{Q.) 



where (G, G') is a 2M-dimensional gaussian centered process with covariance (see (25l) 



,7=1---Af 



Let 



T„ ||2 

Gt + mji„(t) dt, 



= {Gt + m^„ (t))' . dWj - ^ ^ 
y/ = (G; + ni^t))' -dWl^^ 1^ II G; + m4t)||' 



df. 



(36) 
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Then 



exp y/ 
£:e[expy/ 

exp y/ 
e[expy/] 



exp ( y - y/ 



exp 



a(y-y/) 



by Jensen inequality. 

Then, using Holder inequality twice with conjugate exponents {p,q) and {o^rf), one finds: 



exppy/ 



S (£:5[expy/ 



exp |nCTr,^(/i„)|dP®" 



(37) 



We first bound the first term of the right hand side. Let 7^ j^t be a probability measure on O such that 

, _ ni;ii^xp-§ j„^G7^dt ^ 

According to appendix A of [3] (where p = p'^), we have, for any p > 0, that G is a M-dimensional centered 
gaussian process under 7^ ■ 

Consequently, using the independence of (G, G')'s components: 

^^[exppy/] = f[ exp|p(^^m^(t)dVy;" - l^^mf (t)dt)| f5[exp| - |^^Gr"dt}] 
X |exp|p(^^Gr(dVK;° -m°(f)dt))|d7^^^r 

= n {p( m:{t)dw:- mf{t)dt) I £5 [exp I - 1 cr^dtj] 

CiidWt -mZ{t)dt)) 



Hence, 



exp 



exp 




) 


/exp ( 


-l!;^Gfdt)d^, 



d7.)} 



£:e[expy/] i-iifjJexp-i/o^Gr'df] 1 2 7 V io ^ 



exp-f fi^GT^dt , 
J exp-f Jo G^^dtd-y^ 
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Remark that/°(p) is bounded for p > 0. In fact, on one hand it is clear that 

VpG [0,+(»[,fe[exp-|^^ Cr'dt] < 1, 



furthermore Jensen inequality and Fubini theorem give us: 

..[exp{ - ll^Gfdt}] > exp{ - l£s.[Gf]dt} > exp { - ^^}. 

Therefore, bounded convergence monotone gives ,f"(p) — > 1 and, similarly, g°'{p) — > as p \ 1. Moreover, as 
( lo Gt" {dWf" — m"(t)dt)^ has finite moments under 7^, we can find a finite constant Ci{p), Ci{p) \ as 
p \ 1, such that: 

[exppF/] 



cxp y/ 



Ci(p)n 



(38) 



Moreover: 



BS^I^Js. expj^ (Gt + m,(f))'-dWt(a;)-^^ ||Gt + m4t) 

expi^a j\Gt+m,{t)y ■dWt{x)^dP{x) exp | - | ||g 

- — / Gt + m^t) 
2 Jq II I 



dP(x) 



t + m„(t) dt 



exp 



So that if we t ake a close enough to 1, we can find a finite constant C2(cr), limcr^i C2(o-) = 0, such that 
(see inequality ( 30 1 ) : 



exp 



(T^ - (J II l|2 

— - — / \\Gt + m.^m\ dt 



< e 



C2(<T)n 



(39) 



We now will bound the last term of the right hand side of (37 1. By Cauchy-Schwarz inequality, if k = qari 



B3<\[ YlSi^ exp\2^ [ (Gt-G; + (m^„ -m,)(t))'-dW^ 

[J (i,ieB(iy,S) -^-^ I Jo 

1 

-2k^ f ||Gt-G;4{m^i„-m,)(f)fdt|]dP®"i i / £:Jexp|2K'/ |jGt-Gi + ( 



mu„ -m. 



OW dt 



K Gt + m/i„(t) - Gi + mi,(t) 
Jo " " " II 



dP' 



The first term is the square root of a martingale's expectation, thus equal to one. For the second term, we 
remark that: 



^ (G? +m^„(t))' - {Cr + mZ{t)fdt < ^ Q (G? - Gf + {mljt) - mZ){t)ydt 

T 



+ ^ {G7 + Gr + {ml^{t)+mZ)(t)fdt^ 
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so that, by Cauchy-Schwarz inequality: 



j El, exp|(4K'' + K(5-3)^ ||Gf-Gj + (mji„ -m,.)(tW)|j''di| 

j £(, expjft^s^ ||Gt + G; + (m/i„ +m^)(f)|pdt 
< exp^ {l(4.c^ + .5-'^)^^T5^ + (2.5^r^)} 



dP 



X 



exp|2(4K^ + K(5"5)y ||Gt-G;||dt} 
exp|2K55^ ||Gt + G^ll^dtj 



dP^- 



As £5 [/^-^ ||g, - G[f dt] = E*!,=i ^ /o^/ - S^,{y^_^^j'dax,y)dt < Y.",=, 4^5' (in 

fact, dT^fin,^) < 5), we can use appendix A, lemma 3.2 of to bound the two last term of the previous 
inequality: there exists two finite constants Ci (5) and C2 (5) such that, 



exp|2(4K^ + «:(S"3)y^ ||Gt-G;||dt| < exp |4(4k^ + K5"5)Cr(5)CT5^} 



exp 



'^"Gt + G'£dt 



Hence, we can find Ck{S), lim^-^o Ck(S) — 0, such that 

P? < e'^-^*)" 



(40) 



We conclude by using ( 38 1, (39 1 and (40 1 in ( 37 1. 
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